{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一、矩阵\n",
    "\n",
    "矩阵其实就是数的表格，它的加法减法数乘基本没啥好说的，一看就懂。\n",
    "\n",
    "矩阵更高层次的内涵是：**矩阵是线性操作（线性变换）的一种抽象表达**。\n",
    "\n",
    "矩阵运算的重点和难点是 **矩阵乘法** 和 **矩阵的逆（除法）**。\n",
    "\n",
    "其中矩阵的逆，理论价值大于应用价值，数值计算中很少使用矩阵的逆（因为效率低下）。\n",
    "\n",
    "这里就着重强调矩阵乘法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二、从不同的视角去理解矩阵乘法（Perspectives on matrix multiplication）\n",
    "\n",
    "### 2.1 矩阵乘法的定义：行 × 列\n",
    "\n",
    "$$\n",
    "\\begin{pmatrix}\n",
    " -14 &   5 & 10 \\\\\n",
    "  \\color{red}{-5} & -20 & 10 \\\\\n",
    "  -6 &  10 &  6\n",
    "\\end{pmatrix} =\n",
    "\\begin{pmatrix}\n",
    " 2 & -1 & 5 \\\\\n",
    "  \\color{red}{3} &  \\color{red}{4} & \\color{red}{4} \\\\\n",
    " -4 & -2 & 0\n",
    "\\end{pmatrix}\n",
    "\\begin{pmatrix}\n",
    "\\color{red}{1}  & 0 & -2 \\\\\n",
    "  \\color{red}{1} & -5 &  1 \\\\\n",
    " \\color{red}{-3} &  0 &  3\n",
    "\\end{pmatrix}\n",
    "$$\n",
    " $\\color{red}{-5 = 3 \\times 1 + 4 \\times 1 + 4 \\times -3}$ ($A$ 的第二行 ⋅$B$ 的第一列).\n",
    "\n",
    "通用公式为\n",
    "$$\n",
    "c_{ij} = \\sum_{k=1}^n a_{ik} b_{jk}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3×3 Array{Int64,2}:\n",
       " -14    5  10\n",
       "  -5  -20  10\n",
       "  -6   10   6"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = [ 2  -1  5\n",
    "      3   4  4\n",
    "     -4  -2  0]\n",
    "B = [ 1   0  -2\n",
    "      1  -5   1\n",
    "     -3   0   3]\n",
    "C = A * B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "true"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 矩阵乘法按定义的实现\n",
    "\n",
    "function matmul_ijk(A,B)\n",
    "    m,n = size(A)\n",
    "    n2,p = size(B)\n",
    "    if n≠n2 error(\"A 的列数必须等于 B 的行数！\") end\n",
    "   \n",
    "    C = zeros(m, p)\n",
    "    \n",
    "    for i=1:m, j=1:p, k=1:n  # 三层循环\n",
    "       C[i,j] += A[i,k]*B[k,j]\n",
    "    end\n",
    "    \n",
    "    return C\n",
    "end\n",
    "\n",
    "matmul_ijk(A, B) == A * B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个是最枯燥无聊的定义，通过调整计算的顺序，我们可以得到矩阵乘法更直观的描述。\n",
    "\n",
    "而其中尤其重要的，就是是下面的 **行视点**和 **列视点**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 列视点：矩阵 × 列向量（matrix × columns）\n",
    "\n",
    "$$\n",
    "\\begin{pmatrix}\n",
    "  \\color{red}{-14} &  \\color{blue}{5}   &  \\color{orange}{10} \\\\\n",
    "  \\color{red}{-5}  &  \\color{blue}{-20} &  \\color{orange}{10} \\\\\n",
    "  \\color{red}{-6}  &  \\color{blue}{10}  &  \\color{orange}{6}\n",
    "\\end{pmatrix} =\n",
    "\\begin{pmatrix}\n",
    "  2 & -1 & 5 \\\\\n",
    "  3 &  4 & 4 \\\\\n",
    " -4 & -2 & 0\n",
    "\\end{pmatrix}\n",
    "\\begin{pmatrix}\n",
    "  \\color{red}{1}  &  \\color{blue}{0}  &  \\color{orange}{-2} \\\\\n",
    "  \\color{red}{1}  &  \\color{blue}{-5} &  \\color{orange}{1}  \\\\\n",
    "  \\color{red}{-3} &  \\color{blue}{0}  &  \\color{orange}{3}\n",
    "\\end{pmatrix}\n",
    "$$\n",
    "\n",
    "$C$ 的第 $j$ 列，等于 $A$ 乘以 $B$ 的第 $j$ 列\n",
    "\n",
    "即 `C[:, j] = A * B[:, j]`，这就相当于多个 $Ax=b$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Array{Int64,1}:\n",
       " -14\n",
       "  -5\n",
       "  -6"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A * B[:,1]  # A × B的第一列 = C 的第一列"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3×3 Array{Int64,2}:\n",
       " -14    5  10\n",
       "  -5  -20  10\n",
       "  -6   10   6"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[ A*B[:,1]  A*B[:,2]  A*B[:,3] ]  # 结果应该等于 C"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.3.2 用途\n",
    "\n",
    "在线性方程组里，我们将 $Ax$ 解释为对 $矩阵A$ 的列向量的线性组合。这里的 $C[:, i] = A * B[:, i]$ 同样可以如此理解。\n",
    "\n",
    "列视点可用于对 $矩阵A$ 的列向量做线性组合，也就是**做初等列变换**！\n",
    "\n",
    "初等列变换包含下列三种操作：\n",
    "- **加减**：从某列减掉另一列的k倍（或加上）\n",
    "- **交换**：互换某两列\n",
    "- **数乘**：将某列的元素全部乘上某个非零常数。\n",
    "\n",
    "**$A$ 右乘一个变换矩阵 $B$，可以对 $A$ 实现上述的列变换。**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3×3 Array{Int64,2}:\n",
       "  2  -1  5\n",
       "  3   4  4\n",
       " -4  -2  0"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3×3 Array{Int64,2}:\n",
       " -2  5  -1\n",
       " -3  4   4\n",
       "  4  0  -2"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A * [-1 0 0\n",
    "     0 0 1\n",
    "     0 1 0]  # 第一列乘上-1，并交换第二列和第三列"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 行视点：行向量 × 矩阵（rows × matrix）\n",
    "\n",
    "$$\n",
    "\\begin{pmatrix}\n",
    "  \\color{red}{-14} &  \\color{red}{5}   &  \\color{red}{10} \\\\\n",
    "  \\color{blue}{-5}  &  \\color{blue}{-20} &  \\color{blue}{10} \\\\\n",
    "  \\color{orange}{-6}  &  \\color{orange}{10}  &  \\color{orange}{6}\n",
    "\\end{pmatrix} =\n",
    "\\begin{pmatrix}\n",
    " \\color{red}{2} & \\color{red}{-1} & \\color{red}{5} \\\\\n",
    "  \\color{blue}{3} &  \\color{blue}{4} & \\color{blue}{4} \\\\\n",
    " \\color{orange}{-4} & \\color{orange}{-2} & \\color{orange}{0}\n",
    "\\end{pmatrix}\n",
    "\\begin{pmatrix}\n",
    "  1 &  0 & -2 \\\\\n",
    "  1 & -5 &  1 \\\\\n",
    " -3 &  0 &  3\n",
    "\\end{pmatrix}\n",
    "$$\n",
    "\n",
    "$C$ 的第 $i$ 行，等于 $A$的第 $i$行 乘以 $B$。\n",
    "\n",
    "即 `C[i, :] = A[i, :] * B`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1×3 LinearAlgebra.Adjoint{Int64,Array{Int64,1}}:\n",
       " -14  5  10"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A[1,:]'*B  # 应该等于 C 的第一行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3×3 Array{Int64,2}:\n",
       " -14    5  10\n",
       "  -5  -20  10\n",
       "  -6   10   6"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[ A[1,:]'*B \n",
    "  A[2,:]'*B\n",
    "  A[3,:]'*B ]  # 应该等于 C"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "行视点可用于对 $矩阵B$ 的行向量做线性组合，也就是**做初等行变换**！\n",
    "\n",
    "初等列变换包含下列三种操作（和初等列变换完全类似）：\n",
    "- **加减**：从某行减掉另一行的k倍（或加上）\n",
    "- **交换**：互换某两行\n",
    "- **数乘**：将某行的元素全部乘上某个非零常数。\n",
    "\n",
    "**$B$ 左乘一个变换矩阵 $A$，可以对 $B$ 实现上述的行变换。**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3×3 Array{Int64,2}:\n",
       "  1   0  -2\n",
       "  1  -5   1\n",
       " -3   0   3"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "B"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3×3 Array{Int64,2}:\n",
       " 1  -5   1\n",
       " 1   0  -2\n",
       " 0   0  -3"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[ 0 1 0\n",
    "  1 0 0\n",
    "  3 0 1 ] * B  # 交换第一行和第二行，并在第三行加上（原来的）第一行的三倍"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 列 × 行\n",
    "\n",
    ">最重要的是上述的两个视点，这个相对就没那么重要。\n",
    "\n",
    "$AB=C$ 可以看成下列形式\n",
    "$$\n",
    "\\begin{pmatrix}\n",
    "  a_1 & a_2 & ... & a_n\n",
    "\\end{pmatrix}\n",
    "\\begin{pmatrix}\n",
    "  b_1 \\\\\n",
    "  b_2 \\\\\n",
    "  ... \\\\\n",
    "  b_n\n",
    "\\end{pmatrix}=\n",
    "\\sum_{i=1}^n a_{i} b_{i}\n",
    "$$\n",
    "\n",
    "其中 $a_i$ 是 $矩阵A$ 的列向量，$b_i$ 是 $矩阵B$ 的行向量，$a_i b_i$ 得到的是一个 $m × p$ 的矩阵。\n",
    "\n",
    "从这个视点也可发现些有趣的现象：若给 $a_i$ 乘上非零常数 $k$，给 $b_i$ 乘上 $1 \\over k$。$AB=C$ 仍然成立，因为这个改变只会影响上述求和式中的 $a_i b_i$ 项，而该项的结果为 $k a_i \\frac{1}{k}b_i = a_i b_i$，没有变化。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE: 变换矩阵 E 一定是方阵，但是被变换的矩阵 A 就很随意了**。具体而言， $m × n$ 的矩阵 A，m 和 n 可以随意。但是它的行变换矩阵应该是 $m × m$ 的，列变换矩阵应该是 $n × n$ 的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5 子矩阵：分块计算\n",
    "\n",
    "$\n",
    "\\left[\n",
    "    \\begin{array}{c|c}\n",
    "        A_1        & A_2 \\\\\n",
    "        \\hline A_3 & A_4\n",
    "    \\end{array}\n",
    "\\right]\n",
    "\\left[\n",
    "    \\begin{array}{c|c}\n",
    "        B_1        & B_2 \\\\\n",
    "        \\hline B_3 & B_4\n",
    "    \\end{array}\n",
    "\\right]=\n",
    "\\left[\n",
    "    \\begin{array}{c|c}\n",
    "        A_1 B_1+A_2 B_3         & A_1 B_2+A_2 B_4 \\\\\n",
    "        \\hline A_3 B_1+A_4 B_3  & A_3 B_2+A_4 B_4\n",
    "    \\end{array}\n",
    "\\right]\n",
    "$\n",
    "\n",
    "可以看到矩阵分块后，仍然满足矩阵乘法规则。\n",
    "\n",
    "对 $m × n$ 的矩阵 $A$ 和 $n × p$ 的 矩阵 $B$ 而言，**矩阵的分块计算只要求 $A$ 的列的划分要和 $B$ 的行的划分一致**，这样 $A_i B_i$ 这个乘法才能成立。而A的行的划分，和B的列的划分，是很随意的。\n",
    "\n",
    "**用途**：分块最重要的应用，应该是分布式计算/并行计算。此外，某些情况下分块可以减少计算量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A * B = [53 43; 75 43; 35 6]\n",
      "A₁ * B₁ + A₂ * B₂ = [53 43; 75 43; 35 6]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "3×2 Array{Int64,2}:\n",
       " 53  43\n",
       " 75  43\n",
       " 35   6"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 举例（注意非方阵也可以做分块）\n",
    "\n",
    "A₁ = [1 2 7\n",
    "      3 4 6\n",
    "      5 -7 5]\n",
    "A₂ = [1 1\n",
    "      -2 3\n",
    "      7 3]\n",
    "A = [A₁ A₂]\n",
    "\n",
    "B₁ = [3 -4\n",
    "      2 3\n",
    "      6 5]\n",
    "B₂ = [-2 1\n",
    "      6 5]\n",
    "B = [B₁  # B₁ 是 2*2\n",
    "     B₂] # B₂ 是 1*2\n",
    "\n",
    "@show A*B\n",
    "@show A₁ * B₁ + A₂ * B₂  # 应该等于 A*B"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 三、矩阵乘法的性质\n",
    "\n",
    "先定义两个常用方阵\n",
    "1. 对角线元素为1，其他元素全为0的方阵，称为单位矩阵（Identity Matrix，eye），记做 $I$。\n",
    "1. 对单位矩阵做**单次**初等变换（行/列）所得的方阵，叫做初等矩阵。\n",
    "\n",
    "\n",
    "1. 矩阵乘法不具备交换性，$AB$ 一般不等于 $BA$. (特殊情况如 $AA^{-1}=I=A^{-1}A$)\n",
    "1. $IA=AI=A$\n",
    "1. 对单位矩阵做一些初等行(列)变换得到矩阵 E，将同样的行(列)变换应用到 $A$ 上将得到 $EA$（$AE$）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 1.0.3",
   "language": "julia",
   "name": "julia-1.0"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.0.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
